The Association of Master Adaptive Learning With Less Burnout and More Resilience in Obstetrics and Gynecology Residents.

PURPOSE: Medical education should prepare learners for complex and evolving work, and should ideally include the Master Adaptive Learner (MAL) model-meta-learning skills for continuous self-regulated learning. This study aimed to measure obstetrics and gynecology (OB/GYN) residents' MAL attributes, assess associations with burnout and resilience, and explore learning task associations with MAL. METHOD: OB/GYN residents were surveyed electronically at an in-training examination in January 2022. The survey included demographic information, the 2-item Maslach Burnout Inventory, the 2-item Connor-Davidson Resilience Scale, 4 MAL items (e.g., "I take every opportunity to learn new things"), and questions about training and learning experiences. RESULTS: Of 5,761 residents, 3,741 respondents (65%) were included. A total of 1,478 of 3,386 (39%) demonstrated burnout (responded positive for burnout on emotional exhaustion or depersonalization items). The mean (SD) Connor-Davidson Resilience Scale score was 6.4 (1.2) of a total possible score of 8. The mean (SD) MAL score was 16.3 (2.8) of a total possible score of 20. The MAL score was inversely associated with burnout, with lower MAL scores for residents with (mean [SD] MAL score, 16.5 [2.4]) vs without (mean [SD], 16.0 [2.3]) burnout (P < .001). Higher MAL scores were associated with higher resilience (R = 0.29, P < .001). Higher MAL scores were associated with the statement, "I feel that I was well prepared for my first year of residency" (R = 0.19, P < .001) and a plan to complete subspecialty training after residency (mean [SD] of 16.6 [2.4] for "yes" and 16.2 [2.4] for "no," P < .001). CONCLUSIONS: Residents who scored higher on MAL showed more resilience and less burnout. Whether less resilient, burned-out residents did not have the agency to achieve MAL status or whether MAL behaviors filled the resiliency reservoir and protected against burnout is not clear.

Determination, categorization, and hierarchy of content for a pediatric emergency medicine curriculum designed for emergency medicine residents.

Background: Currently, the Accreditation Council of Graduate Medical Education requires time-based pediatric experiences for emergency medicine (EM) residents in both pediatric emergency medicine (PEM) and critical care settings. The American Board of Emergency Medicine has published the Model of the Clinical Practice of Emergency Medicine, which is a list of content an EM resident should learn. However, this list is large and without prioritization and therefore can be difficult to incorporate into time-limited curricula. Objectives: The primary objective of this study was to develop comprehensive categorization of PEM content using an EM lens. The second objective was to suggest a prioritization for the EM learner of the enumerated PEM elements. Methods: We first assembled a comprehensive list of PEM concepts, diagnoses, and procedures that might be taught to EM residents. We then convened focus groups composed of key stakeholders to help formulate content and concept themes important for EM resident training. Once the themes were identified, we divided the list of PEM topics into appropriate themes and then carried out a second round of focus groups expanded to include more diverse expert input for prioritizing the elements of the comprehensive list within each theme. Results: We prioritized 168 important PEM concepts from previous standards and emerging PEM literature among 10 identified themes: the pediatric normal, the bottom-line boil-it-down approach, common presentations, high-acuity pediatric cases and procedures, differences between children and adults, same between children and adults, red flags, infrequency of caring for a child compared with an adult, keep breadth but promote self-directed depth, and triage and disposition. Conclusions: Based on input from stakeholders in EM resident education, we identified key themes within PEM education and created a framework for the hierarchical categorization of PEM content for within an EM residency.

Data sharing and big data in health professions education: Ottawa consensus statement and recommendations for scholarship.

Changes in digital technology, increasing volume of data collection, and advances in methods have the potential to unleash the value of big data generated through the education of health professionals. Coupled with this potential are legitimate concerns about how data can be used or misused in ways that limit autonomy, equity, or harm stakeholders. This consensus statement is intended to address these issues by foregrounding the ethical imperatives for engaging with big data as well as the potential risks and challenges. Recognizing the wide and ever evolving scope of big data scholarship, we focus on foundational issues for framing and engaging in research. We ground our recommendations in the context of big data created through data sharing across and within the stages of the continuum of the education and training of health professionals. Ultimately, the goal of this statement is to support a culture of trust and quality for big data research to deliver on its promises for health professions education (HPE) and the health of society. Based on expert consensus and review of the literature, we report 19 recommendations in (1) framing scholarship and research through research, (2) considering unique ethical practices, (3) governance of data sharing collaborations that engage stakeholders, (4) data sharing processes best practices, (5) the importance of knowledge translation, and (6) advancing the quality of scholarship through multidisciplinary collaboration. The recommendations were modified and refined based on feedback from the 2022 Ottawa Conference attendees and subsequent public engagement. Adoption of these recommendations can help HPE scholars share data ethically and engage in high impact big data scholarship, which in turn can help the field meet the ultimate goal: high-quality education that leads to high-quality healthcare.

Promotion in Place: A Model for Competency-Based, Time-Variable Graduate Medical Education.

PROBLEM: Competency-based medical education is increasingly regarded as a preferred framework for physician training, but implementation is limited. U.S. residency programs remain largely time based, with variable assessments and limited opportunities for individualization. Gaps in graduates' readiness for unsupervised care have been noted across specialties. Logistical barriers and regulatory requirements constrain movement toward competency-based, time-variable (CBTV) graduate medical education (GME), despite its theoretical benefits. APPROACH: The authors describe a vision for CBTV-GME and an implementation model that can be applied across specialties. Termed "Promotion in Place" (PIP), the model relies on enhanced assessment, clear criteria for advancement, and flexibility to adjust individuals' responsibilities and time in training based on demonstrated competence. PIP allows a resident's graduation to be advanced or delayed accordingly. Residents deemed competent for early graduation can transition to attending physician status within their training institution and benefit from a period of "sheltered independence" until the standard graduation date. Residents who need extended time to achieve competency have graduation delayed to incorporate additional targeted education. OUTCOMES: A proposal to pilot the PIP model of CBTV-GME received funding through the American Medical Association's "Reimagining Residency" initiative in 2019. Ten of 46 residency programs in a multihospital system expressed interest and pursued initial planning. Seven programs withdrew for reasons including program director transitions, uncertainty about resident reactions, and the COVID-19 pandemic. Three programs petitioned their specialty boards for exemptions from time-based training. One program was granted the needed exemption and launched a PIP pilot, now in year 4, demonstrating the feasibility of implementing this model. Implementation tools and templates are described. NEXT STEPS: Larger-scale implementation with longer-term assessment is needed to evaluate the impact and generalizability of this CBTV-GME model.

Researching models of innovation and adoption in health professions education.

BACKGROUND: Despite the constant presence of change and innovation in health professions education (HPE), there has been relatively little theoretical modelling of such change, the experiences of change, the ideology associated with change or the unexpected consequences of change. In this paper, the authors explore theoretical approaches to the adoption of innovations in HPE as a way of mapping a broader theoretical landscape of change. METHOD: The authors, HPE researchers with an interest in technology adoption and systemic change, present a narrative review of the literature based on a series of thought experiments regarding how communities and individuals respond to the introduction of new ideas or methods. This research investigates the stages of innovation adoption, from the emergence and hype around new ideas to the concrete experiences of early adopters. RESULTS: When an innovation first emerges, there is often little concrete information available to inform potential adopters, leaving it susceptible to hype, both positive and negative. This can be described using the Gartner Hype Cycle model, albeit with important caveats. Once the adoption of an innovation gets underway, early adopter user experiences can inform those that follow. This can be described using Rogers' diffusion of innovation model, again with caveats. Notably, neither model goes beyond the point of single point-in-time, yes/no, individual adoption. Other approaches, such as learning curve theory, are needed to track uptake and maintenance by individuals over time. SIGNIFICANCE: This expanded theoretical base, while still somewhat instrumentalist, combined with complementary theoretical perspectives can afford opportunities to better explore reasons for variance, volunteerism and resistance to change. In summary, change is complicated and nuanced, and better models and theories are needed to understand and work meaningfully with change in HPE. To that end, the authors seek to encourage richer and more thoughtful research and scholarly thinking about change and a more nuanced approach to the pursuit of change in HPE as a whole.

Considerations on the Use of Neonatal and Pediatric Resuscitation Guidelines for Hospitalized Neonates and Infants: On Behalf of the American Heart Association Emergency Cardiovascular Care Committee and the American Academy of Pediatrics.

Between 0.25% and 3% of admissions to the NICU, PICU, and PCICU receive cardiopulmonary resuscitation (CPR). Most CPR events occur in patients <1 year old. The incidence of CPR is 10 times higher in the NICU than at birth. Therefore, optimizing the approach to CPR in hospitalized neonates and infants is important. At birth, the resuscitation of newborns is performed according to neonatal resuscitation guidelines. In older infants and children, resuscitation is performed according to pediatric resuscitation guidelines. Neonatal and pediatric guidelines differ in several important ways. There are no published recommendations to guide the transition from neonatal to pediatric guidelines. Therefore, hospitalized neonates and infants can be resuscitated using neonatal guidelines, pediatric guidelines, or a hybrid approach. This report summarizes the current neonatal and pediatric resuscitation guidelines, considers how to apply them to hospitalized neonates and infants, and identifies knowledge gaps and future priorities. The lack of strong scientific data makes it impossible to provide definitive recommendations on when to transition from neonatal to pediatric resuscitation guidelines. Therefore, it is up to health care teams and institutions to decide if neonatal or pediatric guidelines are the best choice in a given location or situation, considering local circumstances, health care team preferences, and resource limitations.

Twelve Tips for using Learning Curves in Health Professions Education Research.

Learning curves can be used to design, implement, and evaluate educational interventions. Attention to key aspects of the method can improve the fidelity of this representation of learning as well as its suitability for education and research purposes. This paper addresses when to use a learning curve, which graphical properties to consider, how to use learning curves quantitatively, and how to use observed thresholds to communicate meaning. We also address the associated ethics and policy considerations. We conclude with a best practices checklist for both educators and researchers seeking to use learning curves in their work.

It Takes a Village: Optimal Graduate Medical Education Requires a Deliberately Developmental Organization.

Coaching is proposed as a means of improving the learning culture of medicine. By fostering trusting teacher-learner relationships, learners are encouraged to embrace feedback and make the most of failure. This paper posits that a cultural shift is necessary to fully harness the potential of coaching in graduate medical education. We introduce the deliberately developmental organization framework, a conceptual model focusing on three core dimensions: developmental communities, developmental aspirations, and developmental practices. These dimensions broaden the scope of coaching interactions. Implementing this organizational change within graduate medical education might be challenging, yet we argue that embracing deliberately developmental principles can embed coaching into everyday interactions and foster a culture in which discussing failure to maximize learning becomes acceptable. By applying the dimensions of developmental communities, aspirations, and practices, we present a six-principle roadmap towards transforming graduate medical education training programs into deliberately developmental organizations.

Estimating the Irreducible Uncertainty in Visual Diagnosis: Statistical Modeling of Skill Using Response Models.

BACKGROUND: For the representative problem of prostate cancer grading, we sought to simultaneously model both the continuous nature of the case spectrum and the decision thresholds of individual pathologists, allowing quantitative comparison of how they handle cases at the borderline between diagnostic categories. METHODS: Experts and pathology residents each rated a standardized set of prostate cancer histopathological images on the International Society of Urological Pathologists (ISUP) scale used in clinical practice. They diagnosed 50 histologic cases with a range of malignancy, including intermediate cases in which clear distinction was difficult. We report a statistical model showing the degree to which each individual participant can separate the cases along the latent decision spectrum. RESULTS: The slides were rated by 36 physicians in total: 23 ISUP pathologists and 13 residents. As anticipated, the cases showed a full continuous range of diagnostic severity. Cases ranged along a logit scale consistent with the consensus rating (Consensus ISUP 1: mean -0.93 [95% confidence interval {CI} -1.10 to -0.78], ISUP 2: -0.19 logits [-0.27 to -0.12]; ISUP 3: 0.56 logits [0.06-1.06]; ISUP 4 1.24 logits [1.10-1.38]; ISUP 5: 1.92 [1.80-2.04]). The best raters were able to meaningfully discriminate between all 5 ISUP categories, showing intercategory thresholds that were quantifiably precise and meaningful. CONCLUSIONS: We present a method that allows simultaneous quantification of both the confusability of a particular case and the skill with which raters can distinguish the cases. IMPLICATIONS: The technique generalizes beyond the current example to other clinical situations in which a diagnostician must impose an ordinal rating on a biological spectrum. HIGHLIGHTS: Question: How can we quantify skill in visual diagnosis for cases that sit at the border between 2 ordinal categories-cases that are inherently difficult to diagnose?Findings: In this analysis of pathologists and residents rating prostate biopsy specimens, decision-aligned response models are calculated that show how pathologists would be likely to classify any given case on the diagnostic spectrum. Decision thresholds are shown to vary in their location and precision.Significance: Improving on traditional measures such as kappa and receiver-operating characteristic curves, this specialization of item response models allows better individual feedback to both trainees and pathologists, including better quantification of acceptable decision variation.

Interactive computer-assisted learning as an educational method for learning pediatric interproximal dental caries identification.

OBJECTIVE: We developed a web-based tool to measure the amount and rate of skill acquisition in pediatric interproximal caries diagnosis among pre- and postdoctoral dental students and identified variables predictive for greater image interpretation difficulty. STUDY DESIGN: In this multicenter prospective cohort study, a convenience sample of pre- and postdoctoral dental students participated in computer-assisted learning in the interpretation of bitewing radiographs of 193 children. Participants were asked to identify the presence or absence of interproximal caries and, where applicable, locate the lesions. After every case, participants received specific visual and text feedback on their diagnostic performance. They were requested to complete the 193-case set but could complete enough cases to achieve a competency performance standard of 75% accuracy, sensitivity, and specificity. RESULTS: Of 130 participants, 62 (47.7%) completed all cases. The mean change from initial to maximal diagnostic accuracy was +15.3% (95% CI, 13.0-17.7), sensitivity was +10.8% (95% CI, 9.0-12.7), and specificity was +15.5% (95% CI, 12.9-18.1). The median number of cases completed to achieve competency was 173 (interquartile range, 82-363). Of these 62 participants, 45 (72.6%) showed overall improvement in diagnostic accuracy. Greater numbers of interproximal lesions (P < .001) and the presence of noninterproximal caries (P < .001) predicted greater interpretation difficulty. CONCLUSIONS: Computer-assisted learning led to improved diagnosis of interproximal caries on bitewing radiographs among pre- and postdoctoral dental students.

Evaluation of Polytomous Item Locations in Multicomponent Measuring Instruments: A Note on a Latent Variable Modeling Procedure.

This note is concerned with evaluation of location parameters for polytomous items in multiple-component measuring instruments. A point and interval estimation procedure for these parameters is outlined that is developed within the framework of latent variable modeling. The method permits educational, behavioral, biomedical, and marketing researchers to quantify important aspects of the functioning of items with ordered multiple response options, which follow the popular graded response model. The procedure is routinely and readily applicable in empirical studies using widely circulated software and is illustrated with empirical data.

The fundamentals of Artificial Intelligence in medical education research: AMEE Guide No. 156.

The use of Artificial Intelligence (AI) in medical education has the potential to facilitate complicated tasks and improve efficiency. For example, AI could help automate assessment of written responses, or provide feedback on medical image interpretations with excellent reliability. While applications of AI in learning, instruction, and assessment are growing, further exploration is still required. There exist few conceptual or methodological guides for medical educators wishing to evaluate or engage in AI research. In this guide, we aim to: 1) describe practical considerations involved in reading and conducting studies in medical education using AI, 2) define basic terminology and 3) identify which medical education problems and data are ideally-suited for using AI.

Competency-Based Medical Education in a Norm-Referenced World: A Root Cause Analysis of Challenges to the Competency-Based Paradigm in Medical School.

Competency-based medical education (CBME) requires a criterion-referenced approach to assessment. However, despite best efforts to advance CBME, there remains an implicit, and at times, explicit, demand for norm-referencing, particularly at the junction of undergraduate medical education (UME) and graduate medical education (GME). In this manuscript, the authors perform a root cause analysis to determine the underlying reasons for continued norm-referencing in the context of the movement toward CBME. The root cause analysis consisted of 2 processes: (1) identification of potential causes and effects organized into a fishbone diagram and (2) identification of the 5 whys. The fishbone diagram identified 2 primary drivers: the false notion that measures such as grades are truly objective and the importance of different incentives for different key constituents. From these drivers, the importance of norm-referencing for residency selection was identified as a critical component. Exploration of the 5 whys further detailed the reasons for continuation of norm-referenced grading to facilitate selection, including the need for efficient screening in residency selection, dependence upon rank-order lists, perception that there is a best outcome to the match, lack of trust between residency programs and medical schools, and inadequate resources to support progression of trainees. Based on these findings, the authors argue that the implied purpose of assessment in UME is primarily stratification for residency selection. Because stratification requires comparison, a norm-referenced approach is needed. To advance CBME, the authors recommend reconsideration of the approach to assessment in UME to maintain the purpose of selection while also advancing the purpose of rendering a competency decision. Changing the approach will require a collaboration between national organizations, accrediting bodies, GME programs, UME programs, students, and patients/societies. Details are provided regarding the specific approaches required of each key constituent group.

Competency Standard Derivation for Point-of-Care Ultrasound Image Interpretation for Emergency Physicians.

STUDY OBJECTIVE: Because number-based standards are increasingly controversial, the objective of this study was to derive a performance-based competency standard for the image interpretation task of point-of-care ultrasound (POCUS). METHODS: This was a prospective study. Operating on a clinically-relevant sample of POCUS images, we adapted the Ebel standard-setting method to derive a performance benchmark in 4 diverse pediatric POCUS applications: soft tissue, lung, cardiac and focused assessment with sonography in trauma (FAST). In Phase I (difficulty calibration), cases were categorized into interpretation difficulty terciles (easy, intermediate, hard) using emergency physician-derived data. In Phase II (significance), a 4-person expert panel categorized cases as low, medium, or high clinical significance. In Phase III (standard setting), a 3x3 matrix was created, categorizing cases by difficulty and significance, and a 6-member panel determined acceptable accuracy for each of the 9 cells. An overall competency standard was derived from the weighted sum. RESULTS: We obtained data from 379 emergency physicians resulting in 67,093 interpretations and a median of 184 (interquartile range, 154, 190) interpretations per case. There were 78 (19.5%) easy, 272 (68.0%) medium, and 50 (12.5%) hard-to-interpret cases, and 237 (59.3%) low, 65 (16.3%) medium, and 98 (24.5%) cases of high clinical significance across the 4 POCUS applications. The panel determined an overall performance-based competency score of 85.0% for lung, 89.5% for cardiac, 90.5% for soft tissue, and 92.7% for FAST. CONCLUSION: This research provides a transparent chain of evidence that derived clinically relevant competency standards for POCUS image interpretation.

Using learning analytics in clinical competency committees: Increasing the impact of competency-based medical education.

Graduate medical education (GME) and Clinical Competency Committees (CCC) have been evolving to monitor trainee progression using competency-based medical education principles and outcomes, though evidence suggests CCCs fall short of this goal. Challenges include that evaluation data are often incomplete, insufficient, poorly aligned with performance, conflicting or of unknown quality, and CCCs struggle to organize, analyze, visualize, and integrate data elements across sources, collection methods, contexts, and time-periods, which makes advancement decisions difficult. Learning analytics have significant potential to improve competence committee decision making, yet their use is not yet commonplace. Learning analytics (LA) is the interpretation of multiple data sources gathered on trainees to assess academic progress, predict future performance, and identify potential issues to be addressed with feedback and individualized learning plans. What distinguishes LA from other educational approaches is systematic data collection and advanced digital interpretation and visualization to inform educational systems. These data are necessary to: 1) fully understand educational contexts and guide improvements; 2) advance proficiency among stakeholders to make ethical and accurate summative decisions; and 3) clearly communicate methods, findings, and actionable recommendations for a range of educational stakeholders. The ACGME released the third edition CCC Guidebook for Programs in 2020 and the 2021 Milestones 2.0 supplement of the Journal of Graduate Medical Education (JGME Supplement) presented important papers that describe evaluation and implementation features of effective CCCs. Principles of LA underpin national GME outcomes data and training across specialties; however, little guidance currently exists on how GME programs can use LA to improve the CCC process. Here we outline recommendations for implementing learning analytics for supporting decision making on trainee progress in two areas: 1) Data Quality and Decision Making, and 2) Educator Development.

Frameworks for Integrating Learning Analytics With the Electronic Health Record.

ABSTRACT: The information systems designed to support clinical care have evolved separately from those that support health professions education. This has resulted in a considerable digital divide between patient care and education, one that poorly serves practitioners and organizations, even as learning becomes ever more important to both. In this perspective, we advocate for the enhancement of existing health information systems so that they intentionally facilitate learning. We describe three well-regarded frameworks for learning that can point toward how health care information systems can best evolve to support learning. The Master Adaptive Learner model suggests ways that the individual practitioner can best organize their activities to ensure continual self-improvement. The PDSA cycle similarly proposes actions for improvement but at a health care organization's workflow level. Senge's Five Disciplines of the Learning Organization, a more general framework from the business literature, serves to further inform how disparate information and knowledge flows can be managed for continual improvement. Our main thesis holds that these types of learning frameworks should inform the design and integration of information systems serving the health professions. An underutilized mediator of educational improvement is the ubiquitous electronic health record. The authors list learning analytic opportunities, including potential modifications of learning management systems and the electronic health record, that would enhance health professions education and support the shared goal of delivering high-quality evidence-based health care.

Modeling Diagnostic Expertise in Cases of Irreducible Uncertainty: The Decision-Aligned Response Model.

PURPOSE: Assessing expertise using psychometric models usually yields a measure of ability that is difficult to generalize to the complexity of diagnoses in clinical practice. However, using an item response modeling framework, it is possible to create a decision-aligned response model that captures a clinician's decision-making behavior on a continuous scale that fully represents competing diagnostic possibilities. In this proof-of-concept study, the authors demonstrate the necessary statistical conceptualization of this model using a specific electrocardiogram (ECG) example. METHOD: The authors collected a range of ECGs with elevated ST segments due to either ST-elevation myocardial infarction (STEMI) or pericarditis. Based on pilot data, 20 ECGs were chosen to represent a continuum from "definitely STEMI" to "definitely pericarditis," including intermediate cases in which the diagnosis was intentionally unclear. Emergency medicine and cardiology physicians rated these ECGs on a 5-point scale ("definitely STEMI" to "definitely pericarditis"). The authors analyzed these ratings using a graded response model showing the degree to which each participant could separate the ECGs along the diagnostic continuum. The authors compared these metrics with the discharge diagnoses noted on chart review. RESULTS: Thirty-seven participants rated the ECGs. As desired, the ECGs represented a range of phenotypes, including cases where participants were uncertain in their diagnosis. The response model showed that participants varied both in their propensity to diagnose one condition over another and in where they placed the thresholds between the 5 diagnostic categories. The most capable participants were able to meaningfully use all categories, with precise thresholds between categories. CONCLUSIONS: The authors present a decision-aligned response model that demonstrates the confusability of a particular ECG and the skill with which a clinician can distinguish 2 diagnoses along a continuum of confusability. These results have broad implications for testing and for learning to manage uncertainty in diagnosis.

Educating for adaptive expertise: case examples along the medical education continuum.

Adaptive expertise represents the combination of both efficient problem-solving for clinical encounters with known solutions, as well as the ability to learn and innovate when faced with a novel challenge. Fostering adaptive expertise requires careful approaches to instructional design to emphasize deeper, more effortful learning. These teaching strategies are time-intensive, effortful, and challenging to implement in health professions education curricula. The authors are educators whose missions encompass the medical education continuum, from undergraduate through to organizational learning. Each has grappled with how to promote adaptive expertise development in their context. They describe themes drawn from educational experiences at these various learner levels to illustrate strategies that may be used to cultivate adaptive expertise.At Vanderbilt University School of Medicine, a restructuring of the medical school curriculum provided multiple opportunities to use specific curricular strategies to foster adaptive expertise development. The advantage for students in terms of future learning had to be rationalized against assessments that are more short-term in nature. In a consortium of emergency medicine residency programs, a diversity of instructional approaches was deployed to foster adaptive expertise within complex clinical learning environments. Here the value of adaptive expertise approaches must be balanced with the efficiency imperative in clinical care. At Mayo Clinic, an existing continuous professional development program was used to orient the entire organization towards an adaptive expertise mindset, with each individual making a contribution to the shift.The different contexts illustrate both the flexibility of the adaptive expertise conceptualization and the need to customize the educational approach to the developmental stage of the learner. In particular, an important benefit of teaching to adaptive expertise is the opportunity to influence individual professional identity formation to ensure that clinicians of the future value deeper, more effortful learning strategies throughout their careers.